In this work, we investigate the representation capacity of multilayer perceptron networks that use the sine as activation function - sinusoidal neural networks. We show that the layer composition in such networks compacts information. For this, we prove that the composition of sinusoidal layers expands as a sum of sines consisting of a large number of new frequencies given by linear combinations of the weights of the network's first layer. We provide the expression of the corresponding amplitudes in terms of the Bessel functions and give an upper bound for them that can be used to control the resulting approximation.

我们介绍MR-NET，这是一种用于多分辨率神经网络的一般体系结构，也是基于此体系结构进行成像应用的框架。我们的基于坐标的网络在空间和规模上都是连续的，因为它们由多个阶段组成，这些阶段逐渐增加了更细节。除此之外，它们是一个紧凑而有效的表示。我们展示了多分辨率图像表示以及用于纹理放大和缩小以及抗脉化的应用。

我们引入了一个神经隐式框架，该框架利用神经网络的可区分特性和点采样表面的离散几何形状，以将它们作为神经隐含函数的级别集近似。为了训练神经隐式函数，我们提出了近似签名距离函数的损失功能，并允许具有高阶导数的术语，例如曲率的主要方向之间的对齐方式，以了解更多几何细节。在训练过程中，我们考虑了基于点采样表面的曲率的不均匀采样策略，以优先考虑点更多的几何细节。与以前的方法相比，这种抽样意味着在保持几何准确性的同时更快地学习。我们还介绍了神经表面（例如正常矢量和曲率）的分析差异几何公式。

Landing an unmanned aerial vehicle unmanned aerial vehicle (UAV) on top of an unmanned surface vehicle (USV) in harsh open waters is a challenging problem, owing to forces that can damage the UAV due to a severe roll and/or pitch angle of the USV during touchdown. To tackle this, we propose a novel model predictive control (MPC) approach enabling a UAV to land autonomously on a USV in these harsh conditions. The MPC employs a novel objective function and an online decomposition of the oscillatory motion of the vessel to predict, attempt, and accomplish the landing during near-zero tilt of the landing platform. The nonlinear prediction of the motion of the vessel is performed using visual data from an onboard camera. Therefore, the system does not require any communication with the USV or a control station. The proposed method was analyzed in numerous robotics simulations in harsh and extreme conditions and further validated in various real-world scenarios.

Language modeling, a central task in natural language processing, involves estimating a probability distribution over strings. In most cases, the estimated distribution sums to 1 over all finite strings. However, in some pathological cases, probability mass can ``leak'' onto the set of infinite sequences. In order to characterize the notion of leakage more precisely, this paper offers a measure-theoretic treatment of language modeling. We prove that many popular language model families are in fact tight, meaning that they will not leak in this sense. We also generalize characterizations of tightness proposed in previous works.

After just a few hundred training updates, a standard probabilistic model for language generation has likely not yet learnt many semantic or syntactic rules of natural language, which inherently makes it difficult to estimate the right probability distribution over next tokens. Yet around this point, these models have identified a simple, loss-minimising behaviour: to output the unigram distribution of the target training corpus. The use of such a crude heuristic raises the question: Rather than wasting precious compute resources and model capacity for learning this strategy at early training stages, can we initialise our models with this behaviour? Here, we show that we can effectively endow our model with a separate module that reflects unigram frequency statistics as prior knowledge. Standard neural language generation architectures offer a natural opportunity for implementing this idea: by initialising the bias term in a model's final linear layer with the log-unigram distribution. Experiments in neural machine translation demonstrate that this simple technique: (i) improves learning efficiency; (ii) achieves better overall performance; and (iii) appears to disentangle strong frequency effects, encouraging the model to specialise in non-frequency-related aspects of language.

In this paper, we seek to measure how much information a component in a neural network could extract from the representations fed into it. Our work stands in contrast to prior probing work, most of which investigates how much information a model's representations contain. This shift in perspective leads us to propose a new principle for probing, the architectural bottleneck principle: In order to estimate how much information a given component could extract, a probe should look exactly like the component. Relying on this principle, we estimate how much syntactic information is available to transformers through our attentional probe, a probe that exactly resembles a transformer's self-attention head. Experimentally, we find that, in three models (BERT, ALBERT, and RoBERTa), a sentence's syntax tree is mostly extractable by our probe, suggesting these models have access to syntactic information while composing their contextual representations. Whether this information is actually used by these models, however, remains an open question.

Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to widespread adoption, most LLMs are developed by resource-rich organizations and are frequently kept from the public. As a step towards democratizing this powerful technology, we present BLOOM, a 176B-parameter open-access language model designed and built thanks to a collaboration of hundreds of researchers. BLOOM is a decoder-only Transformer language model that was trained on the ROOTS corpus, a dataset comprising hundreds of sources in 46 natural and 13 programming languages (59 in total). We find that BLOOM achieves competitive performance on a wide variety of benchmarks, with stronger results after undergoing multitask prompted finetuning. To facilitate future research and applications using LLMs, we publicly release our models and code under the Responsible AI License.

在本文中，我们对数值模拟的加速感兴趣。我们专注于高超音速行星再入问题，该问题涉及耦合流体动力学和化学反应。模拟化学反应需要大部分计算时间，但另一方面，无法避免获得准确的预测。我们面临成本效率和准确性之间的权衡：模拟代码必须足够有效地在操作环境中使用，但必须足够准确，以忠实地预测现象。为了解决这个权衡，我们设计了一个混合模拟代码，将传统的流体动态求解器与近似化学反应的神经网络耦合。当在大数据上下文中应用以及它们源于其矩阵矢量结构的效率时，我们依靠它们的力量来实现重要的加速因子（$ \ tims 10 $至$ \ times 18.6 $）。本文旨在解释我们如何在实践中设计这种具有成本效益的混合模拟代码。最重要的是，我们描述了确保准确性保证的方法论，使我们能够超越传统的替代建模，并将这些代码用作参考。

我们将图形神经网络训练来自小工具N体模拟的光晕目录的神经网络，以执行宇宙学参数的无现场级别可能的推断。目录包含$ \ Lessim $ 5,000 HAROS带质量$ \ gtrsim 10^{10} 〜h^{ - 1} m_ \ odot $，定期卷为$（25〜H^{ - 1} {\ rm mpc}）{\ rm mpc}） ^3 $;目录中的每个光环都具有多种特性，例如位置，质量，速度，浓度和最大圆速度。我们的模型构建为置换，翻译和旋转的不变性，不施加最低限度的规模来提取信息，并能够以平均值来推断$ \ omega _ {\ rm m} $和$ \ sigma_8 $的值$ \ sim6 \％$的相对误差分别使用位置加上速度和位置加上质量。更重要的是，我们发现我们的模型非常强大：他们可以推断出使用数千个N-n-Body模拟的Halo目录进行测试时，使用五个不同的N-进行测试时，在使用Halo目录进行测试时，$ \ omega _ {\ rm m} $和$ \ sigma_8 $身体代码：算盘，Cubep $^3 $ M，Enzo，PKDGrav3和Ramses。令人惊讶的是，经过培训的模型推断$ \ omega _ {\ rm m} $在对数千个最先进的骆驼水力动力模拟进行测试时也可以使用，该模拟使用四个不同的代码和子网格物理实现。使用诸如浓度和最大循环速度之类的光环特性允许我们的模型提取更多信息，而牺牲了模型的鲁棒性。这可能会发生，因为不同的N体代码不会在与这些参数相对应的相关尺度上收敛。